Bipartite ranking: risk, optimality, and equivalences

We present a systematic study of the bipartite ranking problem, with the aim of delineating its connections to the class-probability estimation problem. Our study focuses on the properties of the statistical risk for bipartite ranking, which is closely related to the area under the ROC curve: we establish alternate representations of the risk, relate the Bayes-optimal risk to a class of probability divergences, and characterise the set of Bayesoptimal scorers for the risk. We further study properties of a generalised class of bipartite risks, based on the p-norm push of (Rudin, 2009). Our analysis is based on the rich framework of proper losses, which are the central tool in the study of class-probability estimation. We show how this analytic tool makes transparent the generalisations of several existing results, such as the equivalence between four seemingly disparate risks from bipartite ranking and class-probability estimation. A novel practical implication of our analysis is the design of a new family of losses with comparable empirical performance to the p-norm push.